Design thinking teaching and learning in higher education: Experiences across four universities

A growing body of literature highlights the increasing demand on college graduates to possess the problem finding, problem framing, and problem-solving skills necessary to address complex real-world challenges. Design thinking (DT) is an iterative, human-centered approach to problem solving that synthesizes what is desirable, equitable, technologically feasible, and sustainable. As universities expand efforts to train students with DT mindsets and skills, we must assess faculty and student DT practices and outcomes to better understand DT course experiences. Understanding how DT is taught and experienced within higher education can help schools promote student learning and align their training programs with professional, personal, and civic needs. In this study, surveys were completed by 19 faculty and 196 students from 23 courses at four universities. DT teaching and learning was characterized by three DT practices and five outcomes. Statistically significant differences were found by discipline of study and student type (i.e., graduate vs undergraduate), but not by gender or race/ethnicity. These results can be used to inform the development of classroom-based DT teaching and learning strategies across higher education institutions and disciplines.


Introduction
Universities have faced considerable scrutiny in recent years for their apparent failure to adequately equip students with the complex reasoning and problem solving skills thought to be at the core of higher education [1][2][3]. A growing body of literature highlights the demand on a1111111111 a1111111111 a1111111111 a1111111111 a1111111111 following research question: how do faculty and students experience design thinking within higher education courses? To answer this question, we explored the following subquestions: 1. What kinds of DT practices are experienced within higher education courses? In what ways do higher education DT practices align with and differ from other industries?
2. What kinds of outcomes are experienced within higher education courses? In what ways do outcomes of DT in higher education align with and differ from other industries?
3. Is DT a valid construct within higher education teaching and learning?
4. What differences exist between groups, such as discipline (business, design & engineering, other), and student type (undergraduate, graduate)?

Methods
To answer these research questions, we utilized a mixed methods design that combined faculty and student surveys with semi-structured interviews. This manuscript focuses on the survey data only. Additional qualitative and mixed methods findings are detailed elsewhere [25].

Survey development
The survey used in this study was adapted from Liedtka and Bahr, who studied DT practices and outcomes among employees of for-profit, non-profit, and government sectors [24]. Liedtka and Bahr survey items were adapted to align with the context of higher education and additional items were included based on research team experiences. The final survey included 11 items about DT practices, 42 items about outcomes from DT, 11 demographic items (e.g., What is your gender identity?), and 11 course descriptor items (e.g., Was the term "design thinking" explicitly referenced in the course?). All DT practices and outcomes items were prompted by the following stem: Please note how often, as a direct result of this specific course, you observed the following [practices/outcomes] and measured on a scale from 1-Never to 5-Almost Always. The research team reviewed the survey for face validity prior to data collection.

Data collection
Students and faculty at four universities in the southeastern United States were recruited for the study during the 2020-2021 academic year. Purposive sampling was used to identify and recruit participants based on their experience with the research focus [26]. In August 2020, research team members recruited faculty at their home institutions who they knew taught design thinking courses. The email recruitment invited faculty to fill out an initial interest survey about their course, including: course number and title; number of students enrolled in their course; whether their course was open to undergraduate students, graduate students, or both; and number of credit hours associated with the course. The survey also asked faculty for informed consent, contact information, and whether they were interested in participating in an optional semi-structured interview at the end of the semester. Faculty who agreed to participate were also expected to recruit students from their DT course to the study. Toward the end of the Fall 2020 semester, research team members emailed the faculty survey to consented faculty from their home institutions. Faculty participants were also asked to forward study invitations to students enrolled in their DT courses. The email invitations drafted by our research team included information about the study and a link to the student survey. Students interested in participating in the study provided informed consent in the online survey and were also asked whether they wanted to schedule an optional follow-on interview. Faculty participants received three email reminders regarding study invitations to their students before the end of the Fall 2020 semester.

Data analysis
Survey data were first analyzed using descriptive statistics, with continuous variables reported as mean ± standard deviation (SD) and categorical variables reported as frequency (percent). An exploratory factor analysis was conducted to identify DT teaching and learning (DT-TL) constructs using principal components analysis with varimax rotation and Kaiser rule (ie, eigenvalues < 1.0) for student survey items also included in the Liedka and Bahr study [24]. Bivariate correlations were calculated using Pearson rho (r p ) and reliabilities were calculated using Cronbach α. Group comparisons were examined using independent t-tests and one-way ANOVA with Bonferroni post hoc analysis. Parametric statistics were considered appropriate due to normality of data and sufficient sample size. Statistical significance was established at α<0.05. All data analysis was performed in SPSS for Windows, v26 (IBM, Armonk, NY).

Ethical considerations, consent
This project was submitted to the UNC Institutional Review Board (#20-2316), Elon University Institutional Review Board (#21-031), Duke Campus Institutional Review Board (#2021-0168), and North Carolina State University (#23502). The submission was approved or determined to be exempt from further review by each review board according to 45 CFR 46.104. Written consent was obtained electronically from all participants at the start of the survey.

Results
Surveys were completed by 19 faculty and 196 students from 23 courses at four universities. The response rate for faculty was 84.2%. Based on the number of students enrolled in each course, our estimated response rate of students was 20.8%. As seen in Table 1, student participants were predominately white (n = 132, 63.4%), female (n = 126, 64.2%), and majoring in Interdisciplinary Humanities/Social Sciences (n = 105, 53.6%). Eighty-seven percent (n = 170) of students were undergraduate students. Similarly, faculty participants were predominately white (n = 14, 73.7%), female (n = 11, 57.9%), and from Interdisciplinary Humanities/Social Sciences (n = 11, 57.9%).
When asked about the course, most students (n = 149, 76.0%) and faculty (n = 14, 73.7%) indicated that DT was explicitly taught in the course and that they utilized a real-world project (n = 160 (81.6%) students and 13 (68.4%) faculty). On average, students reported spending 69.69% ± 21.65% of their time working in teams. Most students indicated having none or limited DT expertise prior to the course (n = 146, 74.4%) and moderate or extensive expertise after the course (n = 152, 77.5%). Similarly, the number of faculty reporting none (n = 0, 0%) or limited DT expertise (n = 7, 36.8%) prior to the course dropped to 2 (10.5%) after the course. Table 2 provides item-level responses for DT practices in the current study and two related DT studies using the same survey items. On a five-point scale from 1-Never to 5-Almost Always, student and faculty participants reported that they used all 11 of the DT practices with moderate to high frequency in the course. Faculty indicated that they most commonly followed a structured process (4.16 ± 0.69), created prototypes of ideas (4.11 ± 0.66), and emphasized active listening among team to find shared meaning (4.11 ± 0.88). Similarly, students most commonly emphasized active listening among team to find shared meanting (4.38 ± 0.77), followed a structured process (4.16 ± 0.75), and generated a diverse set of ideas (4.16 ± 0.85). Faculty and students executed real world experiments least frequently (3.16 ± 1.26 and 2.96 ± 1.34, respectively). Table 3, factor analyses indicated that DT-TL practices can be broadly characterized by three constructs accounting for 59.03% of the variance: Discovery and Ideation (4.04 ± 0.70, 21.67% of variance); Team Formation and Functioning (4.21 ±0.53, 16.02% of variance); and Prototyping and Experimentation (3.44 ±0.94, 21.34% of variance). Survey items loaded into the same three factors as Liedtka and Bahr [24], with the exception of "Followed a Structured Process," which loaded to Discovery and Ideation in Liedkta and Bahr (2019) instead of on Team Formation and Functioning in our study. Students who reported that DT was explicitly taught in their course more frequently engaged in Discovery and Ideation than those who indicated DT was not explicitly taught (4.16±0.65 vs 3.42±0.72, p<0.001). Students from Business and Design & Engineering disciplines also more frequently engaged in Discovery and Ideation than those from Interdisciplinary Humanities/Social Sciences (4.21±0.53 vs 3.85±0.69, p = .04; 4.26±0.74 vs 3.85±0.69, p = 0.004 respectively). Undergraduate students reported Prototyping and Experimentation more frequently than graduate students (3.49±0.88 vs 3.07±1.24, p = 0.04). There were no differences in DT practices found by race/ethnicity, or gender. Table 4 provides factors loadings for outcomes of DT in higher education courses, which are broadly characterized by five constructs accounting for 63.00% of the variance: Implementation Support (4.00 ± 0.70, 18.51%), Psychological Benefits and Motivation (4.15 ± 0.64, . Students who reported that DT was explicitly taught more frequently experienced Psychological Benefits and Motivation than those who indicated DT was not explicitly taught in their course (4.21 ±0.62 vs 3.85±0.87, p = 0.008). There were no differences in outcomes of DT found by race/ ethnicity, gender, or student type (ie, undergraduate vs graduate). As seen in Table 5, four of the five outcomes factors contained similar items and items that differed. Factor 3, for example, emphasized Relationships in both studies and included the survey item Built new relationships locally that continued after the initial project was completed. However, in the current study, other items in that factor addressed Trust (e.g., Built trust among team members) while other items in the Liedtka and Bahr study addressed Resources (e.g., Expanded access to new resources for individuals and teams.)

As seen in
All correlations between the three DT practice and five outcome constructs were statistically significant, ranging from positive, moderate relationships (r p = 0.33) to positive, very strong relationships (r p = 0.76) ( Table 6). Cronbach alpha exceeded .6 for seven of the eight DT constructs (0.53 � α � 0.89), suggesting that the items used to create each construct demonstrated acceptable internal consistency.

Discussion
This study explored faculty and student experiences with DT in courses from various disciplines within four universities. Given the increasing uptake of DT in higher education and across professional fields, this study is timely and critical for understanding the types of DT practices and outcomes experienced, and ways in which DT in these settings might vary among different industries and stakeholders. This work extends research conducted in workplace settings [24] and compliments studies exploring conceptual frameworks and uses of DT in education [27,28]. Luka, for example, integrated multiple DT models to design an

PLOS ONE
Design thinking in higher education international English course that engaged learners in a four-phased learning cycle of experiencing, reflecting, thinking and acting [27] while Wrigley and Straker described the Educational Design Ladder, which illustrates the organization of a multidisciplinary DT program [28].

DT-TL constructs
The results of this study suggest that DT-TL is a valid construct in its own right within the context of higher education. Specifically, the factor analysis revealed 8 distinct factors with high factor loads and the majority of variance accounted for by the analysis, providing support for content specificity. While the 3 DT-TL practice constructs and 5 outcomes constructs are aligned with those described in other contexts [24], they also embody items differently, which may be attributable to the different processes and contexts associated with student learning in higher education environments. As noted below, the experiences of students and faculty in DT-TL-as indicated by item ratings and construct scores-are likely influenced by interactions with others, constraints of higher education systems (e.g., semester timelines), and disciplinary differences in DT-TL approaches (e.g., business vs humanities). Additional research is needed to understand how and why DT is experienced differently by various stakeholders, and which organizational aspects of these contexts might mediate or support relevant outcomes.

DT-TL practices
Although participants indicated frequently utilizing all DT practices, some were used more than others. Namely, participants indicated engaging in Team Formation and Functioning most frequently, which aligns with previous studies [21,24] and likely reflects DT's commitment to collaborative project-based problem-solving. In contrast, participants engaged in Protyping and Experimentation least frequently, which also aligns with previous studies in higher education [21]. We speculate that a lack of prototyping and experimentation could be a byproduct of dominant approaches to classroom learning that deemphasize the need for experiential and experimental practices and the constraints of a semester-long course (i.e., limited time available to iterate through the full DT process). While there are many student benefits to experiential learning [29], applied learning [30], process-based learning [31], and service-learning [32], they are time-consuming and resource intensive to execute and come with their own limitations and challenges [32]. We recommend faculty consider generating lower-stakes, quicker-paced student learning opportunities to prototype and test in addition to offering consecutive semester-long courses to ensure students are provided opportunities to develop these skills. Compared to research in workplace settings [24], our faculty and students reported a greater frequency of the following front-end practices: followed a structured process, emphasized active listening among team to find shared meaning, and focused problem definition on the  user's perspective rather than the organization's. On the other hand, some practices associated with the later phase of the DT process were reported at lower frequency by faculty and students in our study compared to participants in the Liedtka and Bahr [24] study, including: moved multiple ideas into prototyping, got feedback form users and other stakeholders on prototype, and executed real world experiments.

Outcomes of DT-TL
Recent research suggests that DT can empower students to design desirable, feasible, transdisciplinary solutions that promote practical and sustainable outcomes [22,33]. Our results align with Lake and colleagues [22], who also found that the Quality of Solutions Generated was the most frequently experienced outcome, highlighting the process of engaging students in the DT process as a means for solving complex problems. Not surprisingly, Quality as an outcome had a moderately strong, positive relationship with the practice of Team Formation and Functioning, highlighting the potential benefits of well-designed, high-structured teamwork within courses.
In addition, literature highlights potential cognitive and behavioral benefits of DT, showing positive cognitive and behavioral changes for learning and decision-making [28,34]. Our results align with this literature, suggesting that participants frequently experienced Psychological Benefits and Motivation (e.g., kept people motivated to work on a project to achieve impact). These findings are crucial for educators seeking evidence that DT teaching practices provide students with skills and mindsets for more inclusively and resiliently addressing complex, real world challenges. Both benefits highlight the importance of establishing relevance between student learning and real-world situations through DT pedagogies.
We posit that our outcomes of DT-TL were similar and different from those of Liedtka and Bahr [24] due to our different study populations and contexts. We adapted the survey instrument for faculty and students in higher education institutions involved in semester-long classes whereas Liedtka and Bahr [24] designed it for employees of for-profit, non-profit, and government entities. In our sample, the items on trust (built trust among team members, built trust between problem-solving teams and other stakeholders) loaded alongside other relationship items whereas, in Liedtka and Bahr [24], those items loaded onto its own factor. In academic settings, we often talk about relationship-building and trust together, especially as we discuss community engagement, so this makes sense. In community-based participatory research in particular, relationship-building and trust-building go hand-in-hand [35,36].
We also found that survey items kept people motivated to work on a project to achieve impact, increased a sense of ownership and acceptance of a solution, and increased appreciation for use of data to help drive decisions loaded with the factor on psychological benefits rather than the improved implementation and adaptation factor from Liedtka and Bahr [24]. These three items mark shifts in individual mindsets so this grouping makes sense. Relatedly, the new factor we proposed is termed "Individual Adaptation and Flexibility" and centers around growth mindset and a willingness to learn and change.

DT-TL group differences
Our findings generally indicate that DT-TL practices and valued outcomes are prevalent across disciplines, providing early insights into the potential merits of DT-TL as an interdisciplinary process that is valued across institutions. Several group comparisons warrant discussion and further research, such as undergraduate students experiencing Prototyping and Experimentation more frequently than graduate students. Although graduate degree programs typically immerse graduate students in research, we posit that the structure and processes within graduate programs-which tend to include close oversight from an expert faculty member in a focused area of study-may limit the creativity and brainstorming opportunities afforded to undergraduates. Alternatively, if graduate students experience Prototyping and Experimentation frequently in their degree program, they may perceive this practice as less frequent in DT coursework compared to undergraduate students who are not engaged in similar research programs. As it relates to race and ethnicity, lack of differences should be interpreted with caution since data were collected from predominantly white institutions and the sample sizes for subgroups were relatively small. However, this finding should be explored further given legitimate concerns that DT practices can further privilege those with privileged identities [37,38].

Future directions
This study marks an important step in understanding DT across institutions and disciplines within higher education and across educational and professional divides. Our findings, coupled with those from K-12 education-where DT is lauded as an integral cognitive process involving creation, collaborative sense-making, reasoning with evidence, experimentation, and evaluation [39][40][41]-pose several intriguing opportunities for next steps. Taken together, K-12, higher education, postgraduate, and professional studies contribute to the growing body of research that demonstrates clear DT benefits for students throughout their educational careers and beyond. As education faces ongoing scrutiny about its relevance and value, educators must adopt strategies that enable students to address increasingly complex real-world challenges. In Doctors as Makers, for example, Baruch implores medical curricula to foster creative and critical thinkers that can work through not-knowing, seek compassionate solutions, and explore questions differently within an environment supportive of iterative development [42]. Understanding the ways in which DT compliments traditional problem-solving frameworks, such as the scientific method and clinical decision making, could be an important step toward optimizing how DT-TL is integrated into our curricula [16].
With its emphasis on situated and relational problem-solving frameworks, DT-TL also aligns with "problem-posing" pedagogical theory and praxis that is core to various academic disciplines, such as innovation and entrepreneurship. In Pedagogy of the Oppressed, Freire described problem-posing education as a shift from a content-centric, hierarchical educational model to a contextually responsive, co-creative one [43]. Specifically, DT-TL maps seamlessly onto many of the problem-posing processes taught in innovation and entrepreneurship education, particularly to the discovery process (e.g., problem identification through understanding stakeholder needs) and the development process (e.g., iterative testing and evaluation of prototypes). Similarly, other pedagogical models, such as community-based, project-based, and problem-based-learning include components that align closely with DT. Since institutions who make DT a central strategic focus are more likely to bolster and sustain a competitive advantage [44], more research is needed in higher education to understand the potential prevalence and relationship of DT-TL to pedagogical praxis and other pedagogical models (e.g., problem-based learning) across various disciplines.
Moving forward, we encourage DT-TL educators to extend our work by collecting similar data, developing improved quantitative measures, and implementing longitudinal studies with faculty, students, and project partners. Larger, more representative samples will promote generalizability and enable analyses and sub-analyses examining additional institution-(e.g., university size, Carnegie Classification, institutional control) and faculty-level (e.g., number of years teaching DT, discipline of home department or school, rank or track) factors. Moreover, it would be helpful to know how often students experience DT prior to taking these courses, shortly after taking these courses, and several months or years after taking the courses. This could allow us to better establish causality and attribute changes to the specific courses. Additional research should also be conducted to understand DT-TL outcomes for co-curricular opportunities, such as student organizations and internships to determine what other opportunities in higher education settings can yield similar or better outcomes.

Limitations
With regards to study limitations, we encountered standard challenges with response rates, missing data, and non-response. We recruited faculty and students from four predominantly White higher education institutions in the same state and had a small sample size of faculty at each institution. Data also reflect student and faculty perception at the end of a course-not the long-term outcomes/value of these practices for them or for their project collaborators. That being said, our results are not meant to be generalizable to all DT courses in across higher education settings. Rather, they provide an important first glimpse into DT-TL practices across multiple universities and disciplines.

Conclusion
College students must be equipped with the critical, creative, and collaborative skills needed to address society's increasingly complex and difficult challenges. As this study further verified, DT helps to generate trust across collaborators, fosters the motivation needed to sustain problem-solving efforts, and increases the quality of solutions generated. This study demonstrated the validity of DT across disciplines and universities, the ways in which DT practices and outcomes are experienced in higher education, and a number of important differences between DT practices within higher education and other sectors. Extending this work to include additional universities and research methodologies is critical for providing insight into enhancing the value of DT pedagogies.
Supporting information S1